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Abstract 

We propose a novel estimator of the autocorrelation function in presence of missing 
observations. We establish the consistency, the asymptotic normality, and we derive 
deviation bounds for various classes of weakly dependent stationary time series, in- 
cluding causal or non causal models. In addition, we introduce a modified version 
periodogram defined from these autocorrelation estimators and derive asymptotic dis- 
tribution of linear functionals of this estimator. 

1 Introduction 

The estimation of the sample autocorrelation function (hereafter ACF) from observations 
of Xi, . . . , Xjy is important to understand the process and allows model identification. 

In the classical time series analysis, the innovations (ej)jgz in the linear process (Xj) 
are often assumed to be independent and identically distributed (iid), see for example 
Brockw ell and Davis, 1991] , |Box and Jenkins, 1970| . In this case asymptotic properties 



of the partial sums, especially the sample ACF and the ratio of the sample covariance 
have been extensively studied in the literature. A summary of results about the asymp- 
totical theory of the sample ACF of autoregressive processes can be found for instance in 
Bro ckwell and Davis, 1991] , Chapter 7.2 and 13.3, or Embrechts et al., 1997| , Chapter 



7.3. 

In practice, however, frequently the time series are not fully observed, and there may 
often be substantial numbers of missing values for a variety of reasons. The analysis of 
irregularly observed time series is one of the most important problems faced by applied 
researchers whose data arise in the form of time series. The study of the asymptotic 
properties of the ACF function of a time series model in presence of missing observations 
is more difficult than in the complete case. 

Most of the literature above asymptotic properties of time series with missing obser- 
vations is concerned with linear processes with normal innovations. In addition, these 
perturbations are usually regarded as strict white noise. This assumption is very re- 
strictive; this characteristic implies only linear models with homoskedastic conditional 
variances. As far as we know, the first study that extended the sample ACF to the case 
of missing observations is Parzen, 1963 . Their study formulated that the values of the 
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observed series at unequally spaced times can be represented as an amplitude modulated 
time series Yi = CiX; L where (C$)i e z represents the censoring process. The asymptotic 
properties of this modified ACF were investigated in |Dunsmuir and Robinson, 1981| un- 
der various assumptions on the noise of the linear representation (ej)j 6 z. More recently, 
|Yajima and Nishino, 1999 compare three estimators of the autocorrelation function for 
a stationary process with missing observations. The first estimator is the sample ACF 
extended to the case with censored data proposed originally by |Parzen,T9 63|. The others 
estimators are extensions of this first estimator. The authors derive asymptotic distribu- 
tion for both short memory and long memory models for the three estimators of the ACF 
with missing observations. They impose the same assumptions on the innovations (ej)j e z 
as those in |Dunsmuir and Robinson, 1981] . 

The results obtained for the weak convergence studies for sample ACF in presence of 
missing observation assume asymptotic stationary to fourth order for the (Cj)j g z> then 
the central limit theorem is given by |Dunsmuir and Robinson, 1981 for \^N(jy,n{^) — 



1C,n{£)ix {(■))■ From this, the central limit theorem can be deduced for the y N^x^i^) — 
jx(£)) or VN(px,n(() - px(#)), where Px,n(£) = 7x,aK^)/7x,7v(°) is the lag-£ serial 
correlation and px(£) = 7x(^)/7x(0). In particular, if the (Aj)j £ z are iid with finite 
fourth moment then p~x,n(£) are asymptotically independent normal. 

The asymptotic problem of the sample ACF becomes more difficult if dependence 
among (ej)j G z is allowed. Financial time series often exhibit that the conditional variance 
can change over time, namely heteroskedasticity. Thus, the classical limit theorems cannot 
be directly applied to process with the above condition. 

Theorem 6.7 in |Hall and Heyde, 1980| (p. 188) asserts asymptotic normality of sam- 
ple correlations for martingale differences (ej)j £ z for which E(e?|J r i_i) = a positive con- 
stant. In the literature the above condition is widely used. However, this condition 
appears too restrictive and it excludes many important models. Among them the most 
interesting case is the ARCH model. Thus, limit theorems by |Hall and Heyde~ 1980 



or 



|Dunsmuir and Robinson, 1981) cannot be directly applied to linear processes with ARCH 
innovations. Our results avoid this limitation. 

On the other hand, various generalizations of independence have been introduced in 
order to extend the theory that exists to the independence framework to the more general 
models. The more recent is the notion of weak dependence introduced and developed by 
|Doukhan and Louhichi, 1999| . Our choice is explained by numerous reasons; the frame 
of weak dependence includes large classes of models and can be easily used in a very large 
statistic problems. 

We shall consider the estimator of the ACF in presence of missing observations. The 
asymptotic behavior of the sample ACF is examined for a very general process included 
for the first time process whose innovations are dependent. Central limit theorems are 
established under fairly mild conditions. 

Two frame of weak dependence are considered in this study. The first one exploits a 
causal property of dependence, the #-weak dependence property (see |Dedecker and Doukhan, 20 03 1). 
Under some conditions, the asymptotic normality of the covariance function with missing 
or censored observations is found. The second frame of weak dependence, the A-weak de- 
pendence property (see |Doukhan and Wintenberger, 2007] ), which includes rj and K-weak 
dependences. This notion is convenient for Bernoulli shifts with associated inputs. 

The paper is organized as follows. In Section 2 we introduce the notation and various 
weak dependent coefficients. Section 3 is devoted to limit theorems for causal and non 
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causal weakly dependent time series. Proofs and technical results are given in the last 
section. 



2 Notations and Main assumptions 

Let (Xi)i(zz be a discrete-time second-order stationary time series with (zero- mean). Fol- 
lowing |Parzen, 1963 , we assume that the observations are given by 



Y{ — CjXj, (1) 

where (Cj)j e z is a non-negative modulating process taking values in [0, 1]. When d takes 
values in {0, 1}, the observations are censured, but more general modulations can be 
considered as well. Throughout the paper, this process is assumed to be independent from 
(Xijfez. This property is essential in order to allow recovery of the covariance structure 
of (Xi) ie z- _ 

We denote by Xn,Yn the sample means of (Xi)f =1 and (Yi)f =1 and by 7x,jv(0 
and ^y,n(£) the usual estimates of the covariances Jx(£) = Cov(Xo, Xg) and yy(£) = 
Cov(Y ,Y e ). 

The so-called Parzen estimator of the autocovariance coefficient 7x(^)i is given by 

Similarly the autocorrelation function px{£) is estimated by px,N = 7x,at(^)/7x,jv(0)- 

We study both the consistency and asymptotic normality of the autocovariance and 
autocorrelation functions of time series with missing observations, and also establish non- 
asymptotic deviation bounds. These results are obtained under general dependence struc- 
tures. 

2.1 Weak-dependence measures 

Let p be a positive integer. For / : M p — > M a function, define Lip / the Lipschitz coefficient 
by: 

t - , \f(xi,...,x p )-f(y 1 ,...,y p )\ 
Lip / = sup — : — : i— . 

(x 1 ,...,x p )^(y 1 ,...,y p ) \Xl-yi\-\ + \X p - y p \ 

Definition 1. fDoukhan and Louhichi, 19991 The vector-valued (dxl) process is 



said to be weakly dependent if 

\Cov{f(Z Sl ,.. .,Z Su ),g(Z tl ,.. .,Z tv ))\< 4>(u,v,Lip f, Lip g)e z {r), (3) 

for any real valued functions f and g defined respectively on M. ud and M. vd , that satisfy 
ll/lloo) Halloo < 1 an d Lipf,Lipg < oo, and for any r > and any (u + v) -tuples such 
that s\ < ■ ■ ■ < s u < s u + r < t\ < ■ ■ ■ < t v . Here, the sequence (e^(r))^ is assumed to 
decrease to zero at infinity and ijj : N 2 x (M + ) 2 — >■ M + is a function. 

Specific functions ip yield different notions of weak dependence which have been shown 
to be appropriate to cover various time-series settings |Dedecker et al., 2007| : 

• ip(u,v,a,b) = vb corresponds to the notion of 6*-dependence. 
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• ifi(u,v,a,b) = uvab, corresponds to the notion of K-dependence. 

• i/j{u, v, a, b) = uvab + ua + vb, corresponds to the notion of A-dependence. 

For simplicity the sequence ex(r) will be denoted respectively as 6x(r), *tx(r), and Xx(r). 
We shall also consider strong mixing coefficients aj(r) related with ip(u, v, a, b) = 1; in this 
case heredity is complete through measurable images, see |Rio, 2000| or Doukhan, 1994 



The following simple lemma, which relies on the decomposition of the covariance of two 
random variables conditioned to two independent c-algebra, is intrumental in the sequel: 

Proposition 1. Assume that {Ui)i^% and {Vi)i^% are two vector-valued independent pro- 
cesses. Assume in addition that these two processes are (e, if)) -weakly dependent with the 
same ^-function and sequences denoted (ej/(r)) rg z and {ev{r))r&, respectively. Then the 
vector-valued process (Wi)iez with W% = (Ui,Vi) T is also (ew,tp)- weakly dependent with 
e w {r) = eu(r) + e v (r). 

As a consequence, provided that and (Ci)iez are both (e, -0)-weakly dependent 

processes, then the process 2 x 1— process Z{ = (X{,Ci) also: weak dependences of its 
coordinates are equivalent to that of this process. These coefficients have some hereditary 
properties. For example, heredity through Lipschitz functions is clear and this may be 
extended to locally Lipschitz functions |Dedecker et al., 2007| . 

Proposition 2. Let (U n ) n ^ be a sequence ofM k -valued random variables. Let m > 1. 
We assume that there exists some constant C > such that maxi<j<^ [|C/i[| m < C. Let h 
be a function from M. k to M. d such that h(0) = and for x,y G R fc , there exist a in [l,m[ 
and c > such that 

\h(x) - h{y)\ < c\x - y|(l + \x\ a ~ l + lyl*- 1 ) . 

We define the sequence (V n ) n< =z by V n = h(U n ). Then, if (U n ) n< =z is weakly dependent 
then (V n ) n £z is also weakly dependent and, 



9 v (r) = O (eu{r)^) ; 

( m — a \ 
Ku(r) m + a - 2 J ; 

( m — a \ 
Xu(r) m + a - 2 J . 



Remark 1. For U n = (X n , X n+ i,C n ,C n+ i) the function h(x,x' ,c,c') = cc'{xx' — 7x(^)} 
satisfies the previous assumptions with a = 2. 



3 Limit theorems 

In this section, we study the asymptotic properties of the Parzen estimator of the ACF 
under the different dependence conditions mentioned above. Denote by 

v{i) = E{C C e } and m(£, k, m) = E(C C e C k C m ) . 

As a consequence of Slutsky lemma and the results in Dedecker et alii (2007) we immedi- 
ately derive 
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Theorem 1. Let {X,j)i & i be a real valued, stationary sequence time series of square inte- 
grate observed, with censored data. Let the modulating process (Cj)j e z be a nonnegative 
bounded stationary process. We assume that (Cj)j e z be independent of the process 
Assume either 



• (-Xj)iGZ and (Cj)jgz are strong mixing stationary time series, E|Xo| m < oo for m > 4 
and Y^i>o i m ~ 4a (i) < °°. 

• (Xi)i<=z and (Cj)i £ z are 6-weakly dependent stationary time series, E\Xo\ m < oo for 

1 m— 2 

m > 4 and X^>o i m ~ 4 8 m - 1 (i) < oo. 

• (Xi)i e z and (Cj)j e z a? ~e n-weakly dependent stationary time series, E|Xo| m < oo for 
m > 4, and re(r) = 0(r~ K ) as r — > oo /or re > ^2 + l/(m — 2) J . 

• (Xi)i<=z and (Ci)iez are X-weakly dependent stationary time series, E|Xo| m < oo for 
some m > 4, and A(r) = 0(r~ A ) as r —t- oo /or A > + l/(m — 2) 

Then, under any of these assumptions, 



N^oo 

where 



aj = £ m(£, k,k + £)[k 4 (£, k,k + £) + lx (k + t) lx (k -I)- i\{£)\. (4) 
fcez 

Remark 2. For non-causal dependent sequences (A and re-weak dependence cases), the as- 
sumptions need to be more restrictive and stronger than for the causal #-weak dependence 
case. For this case, indeed, the dependence condition rewrites 9{r) = 0{r~ e ) as r — ^ oo 
for#>H^(l + l/(m-2; 

Remark 3. From |Dedecker et al., 2007| , we know that conditions of Theorem [1] are suf- 
ficient to obtain the weak invariance principle in the #-weak dependence frame. Analo- 
gously, |Doukhan and Wintenberger, 2007] show the same principle in the A and re-weak 
dependence cases. 

By polarization, we simply derive the following extension of this theorem which will 
be used hereafter (the proof is left to the reader). 

Corollary 1. Under the assumptions of Theorem^ for all k 6 N, £\ < ■ ■ ■ < £ k 6 N k , 

VN(y(ti){ixA^)-lx{£i))\ A iVfcCO.E^i,...,^)) 
V ) \<i<k N ^°° 

where T,(£ u . . . ,£ k ) = (°hi.j)x<ij<i k is de fi ned h V 

= Y, M + Zj)l^(£i, k,k + lj) + lx (k + £j) lx {k - £■) - lx {k) lx (k + lj - £■)] . 
fcez 

The following results provide a.s. asymptotic behaviour of the Parzen autocovariance. 



5 



Theorem 2. 



ix,n{£)-ix{£) ^ 0, 



if E|Ao| m < oo for some m = 2 + 5 and, either the processes are 9 -dependent with 

r(<5-l) + l S 

Si>o^ r ~ (1+i) s+1 (i) < oo for some 5 > 0, or they are k, or X-dependent and satisfy 
assumptions from theorem d 

4 Division 

n n 

Let (Ui, Vi)i e z be a stationary sequence and set D n = 1/n iV n = l/n^~] E/iVj then 

i=l i=l 
N n = N = EUxVx, D n = D = EC7i and £ n = iV n /Z) n , R n = R = N/D. 

Theorem 3. Let (Ui,Vi)i£z be a stationary sequence with U > (as.). Let < p < q 

, . , P9 , p(? + 2) 
ana assume that for r = ana s — 



q — p q — p 

[l^iVillr < C, ||Vf|| a < C. 

Assume that the dependence structure of the sequence (t/^V^gg is such that 

\\D n -D\\<-^=, \\N n -N\\ <-^= (6) 

ii ii q n ii up n 



then \\R n - R\\ p = O (l/Vn) . 

In the following cases, we assume that \\Vi\\ s < c and ||?7jVj|| r < c and prove that 
(|SJ) holds. Denote = f/jV^ — EUVi. For simplicity we will often assume ||C/j||oo < oo, 

\\Vi\\ r < OO. 

4.1 Independent case 

Assume that (Ui,Vi) is i.i.d. Assume that ||£7o||g — c an d H^o^bllp < c. From the 
Marcinkiewikz-Zygmund inequality for independent variables, for 2 < q < r, we get 



E 
E 



D n -D 

Nn-N 



9 < QE|C/i| 9 n~2 < Cn'i, 
P < C p ^\UiVi\ p n-2 < Cn~2, 



and ([6]) holds. Now Holder inequality implies those relations if ||£/oL; and [|Vq|| vp < oo. 

q-p 

4.2 Strong mixing case 

Denote (ai)ieN the strong mixing coefficient sequence of the stationary sequence (U, Vi)^. 

Proposition 3. Assume that for r' > q, \\Uo\\ r i < c. Relation holds if on = 0(i~ a ) 
with 

' p r \ I q r' 
a > I £ V ' 



2 r — p J \2 r' — p J ' 
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4.3 Causal weak dependence 

Define the 7 coefficient of dependence of a centered sequence (Wj)ieN with values in M. d 
by 

7i = sup||E(Wi+ fc |>tfc)|| 1 

fc>0 

Proposition 4. Assume that ||£/o||oo < c. Relation fifty holds if the sequence of coefficients 
7 associated to the stationary sequence (Wj)ieN = ^4)ieN suc/i t/iai 7, = 0(z~ 7 ) mii/t 

/p r — l\ q 
7 > V -. 

4.4 Non causal weak dependence 

Here we consider non causal weakly dependent stationary sequences of bounded variables 
and assume that q and p are integers. A sequence (Wj)j 6 N is said to be A-weakly dependent 
if there exists a sequence (A(i))jgN decreasing to zero at infinity such that: 

Cav(gi(Wi 1} . . . , W iu ),g 2 (Wj 1 ,. . . , W jv )\ < (uLipg 1 + vLipg 2 + uvLip gihipg 2 ) X(k), 

for any u-tuple (ii, . . . , i u ) and any u-tuple (ji, . . . ,j v ) with i\ < ■ ■ ■ < i u < i u + k < j\ < 
••• < jv where gi,g 2 are two real functions of A^ = {gi G A| ||<?i||oo < 1} respectively 
defined on ~K Du and M. Dv (u, v G N*). Recall here that A is the set of functions with 
Lip g\ < 00 for some u > 1, with 

t ■ lsi(yi> • • • ,y«) - 31 • • • 

Lip^l = SUp : : : : . 

( X1 ,...,x u )^( yi ,...,y u ) \yi-xi\-\ v\y u -x u \ 

The monograph Dedecker et al. (2007) Dcdcc ker et al., 2007| details weak dependence 
concepts, as well as extensive models and results. 

Proposition 5. Assume that p and q > 2 are even integers. Assume that the stationary 
sequence (Ui,Vi)i^n is X-weakly dependent. Assume that Zq = UqVq —WJqVq is bounded 
by M. Relation fjf holds if X{i) = 0(i~ x ) with X > f. 



Remarks 

• Unbounded random variables may also be considered under an additional concentra- 
tion inequality (P(Zi G (x,x+y)) < Cy a for some a > 0) and Theorem 3 and Lemma 
1 from Doukhan and Louhichi (1999) Doukh an and Louhichi, 1999| , imply that the 

same relation holds if E\Zi\ q+s < 00, sup x jP(Zj G (x,x + y)) < Cy a , (Vy > 0), and 

00 

Eq + <5 a 
n q ~S l \2 + a ( n ) < oo. 

71=1 

• Non-integer moments q G (2, 3) are considered in Doukhan and Wintenberger (2007) 
(see |Doukhan and Wintenberger, 2007] , Lemma 4), and the same inequality holds 
if ¥\Zi\ q < 00 with q' = q + 5, and X(i) = (D(i~ x ) with A > 4 + 2/q' for q small 
enough: 

q < 2 + ^ (VW + 4 " 2^) 2 + 4(A - 4)(g' - 2) - 2 + q' + 4 - 2A) (<</). 
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5 Spectral estimation 

In this section we study estimation of functionals of the spectral density function from the 
censored time series. Using the Parzen estimator of the covariance 7x,jv(^)j estimates of 
the spectral density of (Xj)j e z can be constructed. More precisely, we introduce a modified 
periodogram defined with the empirical covariance jx,n(£) of the censored process 

Assume now that we wish to estimate linear functionals of the Parzen peridogram, 

Jx(g)= / + %(A)/x(A), 

J —IT 

where fx(X) is the spectral density of (-Xj)jgz. Using the estimates of the covariances 
of (.Xj)j e z given by jx,n(£), estiamtes of the integrated periodogram of can be 

constructed by 

Jx, N (g) = f * g(\)I N (\)d\. 

J — IT 

Under general conditions the integrated periodogram is a consistent estimator of the Jx{g) 
provided the spectral density fx{X) is well defined. 

Our aim is to study the asymptotic behavior of E|Jx-,jv(s) — Jx(g)\ q - We consider the 
Sobolev space T-L s for s > 1: 

H s = {g G h 2 [-ir,ir];g(-x) = g(x), \\g\\ 2 He < oo}, with \\gf Hs = £(1 + \l\) S \ge\ 2 . 

for g a 27r-periodic function such that g G L 2 ([— 2-7T, 2tt[) and g{\) = X^gz5^ e * tt - 
The norm of the dual space H' s of H s writes 

\\T\\ 2 H ,= sup |T( 5 )| 2 = ^(l + |£|)-|T(e^)| 2 . 
Il9lk s <i ee z 

Note that Jx,x{g) and Jx(g) G T~L' S - We have 

Theorem 4. Let (Xj)jgz is a real valued time series observed with censored data such that 
Yli&zlx^) 2 < oo and {ni{iij,k))ij^ the fourth cumulants of X exist. If assumptions of 
Lemma 2 are satisfied, them \/g G % s 

lim E\\Jx, N (g)- Jx(g)\\ 2 n > — >0. 

N— >oo s 

Theorem 5. Under assumptions of Theorem ??, the central limit theorem is satisfied if 
we ssume that \\^c,n{^) — lc(@)\\q < v n for some q > 2, and ||7y,jv(^) — TrCOIh < v n . 
Moreover if ||XjXj + ^CjCj+^|| r < K , and \\XiXi + (\\ s < k. Then 

\fN[J x ,N{9) ~ Jx{g)] A N{0,a 2 {g)) in the space H' s . 

7V->oo 
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6 Technical results and proofs 



Our proof for central limit theorems is based on a weak invariance principle under weak 
dependence conditions. 

6.1 Proof of Theorem [j] 

To study the asymptotic behavior of ^x,n{£) = 7y,Jv(^)/^tv(^) we decompose the quantity 
of interest: 



v N {t) 

= ■ VN - -y x (e)u N (£)} . (7) 

The second factor in the RHS of the previous expression is handled by using a Central 
Limit Theorem for weakly dependent sequences. 

N-e 

^/N{jY,N(i) - 7x(l)m{t)} = N- 1/2 Z ^ (8) 

k=l 

with Z k = CkCk+e (X k X k+ £ — 7x (•£)). According to Propositions CD and [2j (Z k ) is weakly 



dependent. For the k and A-dependence. Dedecker et al., 2007, Theorem 7.1, Theo- 



rem 7.2, p. 154] prove a C.L.T. for k and A-weak dependent processes, and the proof 
is immediate. The situation is a bit more intricate for ^-dependence. We shall show 
that SV = V~N(jx,n(£) — 7x(^)) is asymptotically normally distributed by showing that 
N -1 satisfies a central limit theorem for #-weak dependent variables. Theorem 2 

of |Dedecker and Doukhan, 2003| states that if Sn = X^ 1 z k be a strictly stationary 
sequence of square integrable an centered random variables, then if the condition 

ZqK(Sn I -Mo) converges in Li; (9) 

holds, where Mi = cr(Zj;j < i), then N~*' 2 Sn will be asymptotically normally dis- 
tributed. 

Alternatively, by Corollary 1 of |Dedecker and Doukhan, 2003| , in the #-weak dependence 
frame, D(2;Xq) implies © where 

r\\x\\i , 

D(p, X) : / {6- l {2u)f- l Q p x 1 o G x {u)du < oo. (10) 
J o 

and for X a real valued random variable, Qx denotes the generalized inverse of the tail 
function x \-t P(|X| < x), and Gx the inverse of x h-> Jq Qx(u)du. |Dedecker et al., 2 007 



Corollary 7.6] gives sufficient conditions to satisfy the condition D(p;X). In particular, 
if p = 2 and ||-X"o||4+<5 < 00 then ||.Zo||2+<5 < oo and Y^i>o^ S (.^x(^)) < 00 f° r some 
5 > 0, (J9j) condition is satisfied and therefore the asymptotic normality in (JHJ follows from 
Dedecker et al., 2007j Corollary 7.5]. 

It remains to show that the limiting covariances are given by a\ = ^ fcgZ EZo^fc- 

aj = E(C C e C k C k+e )[E(X X e X k X k+e ) - (11) 
fcez 
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We use the following identity for k) G T? for simplify the expression (1111) : 

K 4 (i,j, k) = EX XiXjX k - EX XiEXjX k - EX XjEXiX k - EXoX^XiXj 

Then the following expression, which exists for all finite I, is equivalent to the asymptotic 
variance in equation (lllj) 



a\ = J2E(C C e C k C k+e )[K 4 (t, k,k + £) + lx (k + £)jx(k -I)- 
kez 

We note that when all the (Xj)j e g are observed E(CoCiC k C k+ g) = 1 so that of agrees 
with |Rosenblatt, 1985| , Theorem 3, p. 58. 



Finally the first factor in ([7]) converges in probability to v{£) because the stated assump- 
tions ensuring a central limit theorem also imply the convergence of the empirical variances. 
Slutsky's Theorem allows to complete the proof. This completes the proof of Theorem 
[Q □ 

Corollary [T] is standard from the previous result. 
6.2 Proof of Theorem [2] 

To obtain strong laws for the sample autocorrelation function of (Xi)i<=z estimated like 
a function of (ii)iez and (C^i^% in the #-weak dependence frame, by theorem 3 of 
|Dedecker and Doukhan, 2003| if D(p, X) holds for some p G [1, 2[, then n _1//p J27=ii^i ~~ 
E(Xj)) converges almost surely to as n goes to infinity. We use again Lemma 2 of 
|Dedecker and Doukhan, 2003| , we need considered ap> 1 and therefore, we need weaker 
moments conditions. If ||Xo|| r < oo for some r > 2(l+t5) and ^ i>Q i( r ( 5 ^ 1 )+ 1 )/( r_ ( 1+<5 ))0 i < 

oo for some 5 > 0, then is a sufficient condition for Jx,n(&) — lx{£) 0. 

For the strong laws for A or K-weak dependence cases, |Doukhan and Wm tenberger, 2007 



proved a bound of the (2 + <5)-moment of the sum of a process A or fc-weak dependence. 
This bound it directly yields the strong law of large numbers using the Borel-Cantelli 
lemma. 

6.3 Proof of Theorem H 

In order to prove that \\Jx,n{q) — ^(<?)||^' converges, we use a bound for E\\Jx,n{9) — 
J(g)||^, under further conditions. Then 

\\Jx,n(9)- J(g)\\k = \\J2&xMt)--yx(z))9i\\k 

= II $> + \£\)- s (jx,nW - 7*W)(1 + \t\)'9t\&> 
taz 

< (E(! + \Z\r 2s ("lx,N(Z) - lx(l)f) fee 1 + 

< \\g\\u 2s fe(i + l^l)" 2s (7x,7vW - ix (£)) 2 ) 



? s 9i 
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Therefore, to prove a bound for E|| Jx,n(q) — J (9) ||%' it sufficient to show that E(7x,tv(^) - 
lx{£)) 2 is bounded. 

EfeivG;) - J(g)\\ 2 Wa < \\gWu*. fo 1 + KI)" 2s lE(7x,7vW - lx(£)f 



And, 



/ 1 K v%(Nk) 



lc(i) icW lc(t) 



a+1 



max |AjAj + ^| • 



KKn + ' 2 ~ l<i<n 



< — '— 1| max \XiX i+ t\\\ 2 a\\d - d\ 1+a \ 



2b 



1 „ .„„ ,o„\ 2a 



— — E max X-X- +/ r a v, 



2a \ „,l+a 



|d| Q V l<i<n 

if g > 26(1 + a) or equivalently g — 2(1 + a) > g/a. We need an argument of |Pisier, 1978| 
written as follows: assume that ip : R + — > M + is convex and non decreasing then 

ip (EmaxlXiXi+^A < E^ fmax|X;X;^| 2 ^ < E E 9? (|XX i+£ | 2a ) < ^Eip [\XiX i+e \ 

Hence Emax; \XiXi + e\ 2a < (nc) 2a / s with <p(x) = x s / 2a . Now the bound in the right 

hand side of (fT2|) can be specified as v}^ a {nk) ~i j\d\ a if s > pa this holds because 1 — 
2 12 

— (1 + a) > — > — , if a > is small enough with i > 3dS _|_ I. "W e thus obtain 
q a s z g s 

7cW|g§ - 7x(i)\\ 2 < v n + f v n + ^+ a (nA:)V|d| a , that implies the result of the 
Theorem. □ 

6.4 Proof of Theorem [4] 
1. 

JN(g) - J(g) 



E ixMQsi 


- E 7x W^ 




\e\<N 


££N 




E 1X,n(£)91 


- E 7X,( i )9£ 


- E t*w^ 


\£\<N 


\£\>N 


|£|<JV 


- E "lxif)9l 


- E few - 




\1\>N 






- E 7x(£)ge 


- E few - 


^lx(i))9i 


\£\>N 


\£\<N 




-Ti + T 3 







Ti = E 

n = EfeW- E ^W)^ 



K|<iV 



11 



milk < £(i + i^ir s 7xw 2 <^ E^) 2 = ^<°° 

|^|>iV |£|>Af 

\£\<N \£\<N 



\\Jn(9) - J(9)\\ 2 n/s < 2(11^11^ +E\\T 3 f n/ J 

= |(^+ E( 1 + i^ir s (^+27)) 

K|<jv 

= |(7 + c s (l + K|)- s (^ + 2 7 )) 



2. 



= nE(^(7x,7vW-7xW)5W) 2 

|£|<m 

= nVar(£(7x^W- 7 xW)5W) 



|£|<m 

= n £ Cov(((7x,ivW-7xW)5W),((7x,iv(A:)-7x(A ; ))5(fc))) 

Let A(fc) = (7x,jv(A:) - 7x(fc)) and = (tx.atW - 7 x(^)), then 

ai = n ^ Cov(A(fc),A(£))$(fc)$(£) 

|£|<m 



3. 

B = ^ (I + \k\) 2s Yav {A(k))^2{l + \£\) 2s \ai {A(£)) < oo 

\k\>m I 

Let g : 7^ — >■ -E^, where g denotes the orthogonal projection on the closed linear subspace 
E k C H' s , generated by (e e ) WgeqL with e e {\) = e liX . 
Suppose that g G E k = {g% = 0, \i\ > k}. Then 

VN(J x , N (g) - Jx(g)) A JV(o,<t 2 ( 5 )) 

because is equal to X)|i|<fc3i^JV(i)- 

Also note that if g G and g fc is the projection on Ek, then a 2 (g^) — >■ cr 2 (g) and 
therefore 

ViV[Jx,Jv(5)-«/x(5)] A iV(0,a 2 ( 5 )) V 5 G^' S . □ 

iV->oo 
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7 Examples 

We present here examples where weak dependence, as defined in Section \2.1\ holds. First, 
we focus on some general classes of processes before and then we will study specific models. 
A generic sample follows 

Definition 2. Let (et)tez be a sequence of real-valued random variables and let F : M z — > E 
be a measurable function. The sequence (X t )t<=z defined by 

X t = F(e t - j ;j€Z) (13) 

is called a Bernoulli shift. 

The class of Bernoulli shifts is very general. It provides examples of processes that are 
weakly dependent but not mixing (see Rosenblatt, 1985 1). 



7.1 Markov processes 

Markov processes can be represented as Bernoulli shifts. Consider an IR^-valued Markov 
process, driven by the recurrence equation 

X t = f(X t - U e t ) (i€Z) (14) 

for some i.i.d. sequence (et)teZ with E(eo) = 0, tt independent of {X s ;s < t} and 
/ : R D x R -)■ R D . Then the function F in (jEJ) is defined implicitly (if it exists) by the 
relation 

F(x) = f(F(x'),x ) 
where x = (xq, xi,X2 ■ ■ ■) and x' = (x%,X2, £3, . . . ). 

Assume now in representation (fl~4"|) that xq is independent of the sequence (et)t<=N- 
Suppose that, for some < Cj < 1, 

d 

E|/(0,ei)| < 00 and E|/(u, e x ) - /(«, ei)| < J2 Vi\, (15) 

i=l 

d 



c = q < 1 for all u, v G 



Uj \ ± iui an u, » c » • 

i=l 

Under condition (I15p the Markov process (Xt)tew has a stationary distribution /i with 
finite first moment. Assume now in addition that xq is distributed with fi, that is, the 
Markov chain is stationary. Then, if ([15p holds, such a Markov chain is 0-weak dependent 
with 9 r = c t E\xq\. 

7.1.1 Nonparametric AR model. 

Consider the real-valued functional (nonparametric) autoregressive model 

X t = r(X t ^) + e t , (16) 
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where r : R — > R and (et)tez as in (|14|) . This a special example of a Markov process as 
in (|14p . Assume that |?"(u) — r(u')| < c|n — u'| for all u,u' E R and for some < c < 1, and 
-E|eo| < oo. Then (I15D with D = 1 holds and implies #-weak dependence with 6 r = c r E|Zo|. 

Here is important to note that the marginal distribution of the innovations et can be 
discrete. In such a case, classical mixing properties can fail to hold. For example, consider 
the simple linear AR(1) model, 

X t = 0AVi + e t = <P e t-j, \<P\ < 1- 
i>o 

Let (et)tez be a sequence of i.i.d. Bernoulli variables with parameter s = P[ej = 1] = 
1 — P[et = 0]. The AR(1) process (X t )t<=z with innovations (ttjteZ an d AR parameter 
e g]0, |], is 0-weak dependent with # r = c/> r E|A"o|, but it is known to be non-mixing. In 
this context concentration holds. For example, Xt is uniform if s = \ and it has a Cantor 
marginal distribution if s = | . Hence, without a regularity condition on the marginal 
distribution of eo, Bernoulli shifts or Markov processes may not be mixing. 

7.1.2 Nonparametric ARCH model. 

Consider the real-valued functional (nonparametric) ARCH model 

X t = s(X t ^)e t , 

where s : R — )■ R + and (et)t^z is defined as in (|14p with S|eo| 2 = 1. This is a special 
example of a Markov process as in ()14[) with f(u, v) = s(u)v. Assume that \s(u) — s(u')\ < 
c\u — u'\ for all u, u' G R and for some < c < 1. Then f)15|) with D = 1 holds and implies 
6-weak dependence with 9 r = crE|Ao|. Again, the innovation distribution is allowed to be 
discrete. 

7.1.3 Nonparametric AR-ARCH model. 

We interest now in the combination of AR and ARCH models. This new process have 
nonparametric conditional mean and variance structure, 

X t = r(AVl) + 8(X t -i)et, 

with r(-),s(-) and (etjtez as i n the examples above. Assume the Lipschitz conditions 
on r(-) and s(-) with constants c r and c s , respectively. If c r + c s = c < 1, the process 
satisfies #-weak dependence with 6 r = ct¥.\Xq\. 

7.1.4 Bilinear model. 

We consider the simple bilinear process with the following recurrence equation 

X t = aX t -i + bX t -ie t -i + e t , 

where {et)t& is as in (fl~o]h Such causal processes are associated with chaotic represen- 
tation with stationary 

oo j 

F ( u ) = y^ U 3 Y\_( a + bu s), U = (U ,U1,U 2 , ■■■)■ 

3=0 8=1 
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If the process is stationary and c = E|a + 6eo| < 1, the process satisfies #-weak depen- 
dence, with 8 r = ° l^} ■ 
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